Bringing convolutional neural networks (CNNs) to your industry--whether it be medical imaging, robotics, or some other vision application entirely--has the potential to enable new functionalities and reduce the compute requirements for existing workloads. This is because a single CNN can replace more computationally expensive image processing, denoising, and object detection algorithms. However, in our experience interacting with customers, we see the same challenges and difficulties arise as they move an idea from conception to productization. In this article, we'll review the common challenges and address some of the solutions that can smooth over development and deployment of CNN models in your edge AI application. We see a lot of companies attempting to create models from the ground up.

The once untouchable semiconductor sector continues to attract fervent attention from VCs. The latest news this morning is that Ajay Royan of Mithril Capital has led a $55 million Series D round of financing into Flex Logix, which builds chips designed to bring AI workflows to the compute edge. That follows on earlier rounds in the company totaling $27 million from the likes of Lux, Eclipse Ventures and Tate Family Trust, the investment vehicle of the company's founder and CEO Geoff Tate. The firm previously backed Nuvia, a promising entrant in the server chip market which was founded by several of the top chip designers of Apple's A-line of processors. Mithril invested $240 million in Nuvia last September, just a few months before the company flipped over to Qualcomm in a $1.4 billion transaction announced in January.

Artificial intelligence (AI) and machine learning (ML) are progressing at a rate that is outstripping Moore's Law. In fact, they now are evolving faster than silicon can be designed. The industry is looking at all possibilities to provide devices that have the necessary accuracy and performance, as well as a power budget that can be sustained. FPGAs are promising, but they also have some significant problems that must be overcome. The graphics processing unit (GPU) made machine learning (ML) possible. It provided significantly more compute power and had a faster connection to memory than the CPU.

Deep learning has a DRAM problem. Systems designed to do difficult things in real time, such as telling a cat from a kid in a car's backup camera video stream, are continuously shuttling the data that makes up the neural network's guts from memory to the processor. Some systems need four or even eight DRAM chips to sling the 100s of gigabits to the processor, which adds a lot of space and consumes considerable power. Flex Logix says that the interconnect technology and tile-based architecture it developed for reconfigurable chips will lead to AI systems that need the bandwidth of only a single DRAM chip and consume one-tenth the power. Mountain View-based Flex Logix had started to commercialize a new architecture for embedded field programmable gate arrays (eFPGAs). But after some exploration, one of the founders, Cheng C. Wang, realized the technology could speed neural networks.

This new core has been specifically designed to enhance the performance of deep learning by 10X and enable more neural network processing per square millimeter. Many companies are using FPGA to implement AI and more specifically machine learning, deep learning and neural networks as approaches to achieve AI. The key function needed for AI are matrix multipliers, which consist of arrays of MACs (multiplier accumulators). In existing FPGA and eFPGAs, the MACs are optimized for DSPs with larger multipliers, pre-adders and other logic which are overkill for AI. For AI applications, smaller multipliers such as 16 bits or 8 bits, with the ability to support both modes with accumulators, allow more neural network processing per square millimeter.